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General Achievement Trends — Massachusetts 

K-12 enrollment — 968,661 



The raw data used to develop these state profiles, including data for additional grade levels and years before 2002, can be found 
on the CEP Web site at www.cep-dc.org . Click on the link on the left for No Child Left Behind. In the Document Library, look for 
the most recent report on student achievement since 2002. Below the name of the report, click on the link for View State Profiles 
and Worksheets. Scroll down the page, and click on the Worksheet links for any state. 



Overall Achievement — Key Findings 
Massachusetts Overall Achievement — Key Findings 
General results 

The tables in this profile present state test results in reading and math at three achievement levels (basic, proficient, and advanced) and at one 
grade each at the elementary, middle, and high school levels. These data are more complete than the percentage of students scoring proficient 
that is the main indicator used to determine adequate yearly progress under the No Child Left Behind Act. 

In general, Massachusetts students made gains at all three achievement levels. The exceptions were in reading, where there were slight 
decreases at the basic and proficient levels in the elementary grade analyzed. At the advanced level, there were moderate-to-large gains across 
the board in math, especially at the high school level. 

Specific results 

• Between 2002 and 2008, the percentage of students performing at the basic level and above in reading decreased slightly at the 
elementary grade analyzed but increased at a moderate-to-large pace at the high school level. (In middle school reading, the percentage 
basic stayed the same over the three years with comparable data, 2006 through 2008). In math, the percentage of students at basic and 
above increased at a moderate-to-large rate at all three grade levels analyzed. 

• Between 2002 and 2008, the percentage of students scoring at the proficient level and above in reading went down slightly at the 
elementary school grade analyzed but rose at a moderate-to-large rate at the high school grade analyzed. In middle school reading, 
comparable data were available for only three years, 2006 through 2008; during this period, the percentage proficient increased slightly. In 
math, the percentage proficient grew at a moderate-to-large rate at all three grade levels analyzed between 2002 and 2008. 
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• The percentage of students reaching the advanced level in reading increased slightly at the elementary and high school grades analyzed 
between 2002 and 2008. In middle school reading, there was no change in the percentage advanced between 2006 and 2008. In math, 
the percentage of advanced students rose at a moderate-to-large rate at all three grade levels analyzed. At the high school level, a 
notable 42% of students had reached the advanced level in 2008, up from 20% in 2002. 



Data Limitations 



Years of comparable percentage proficient data 

Years of data needed to compute effect sizes 
Disaggregated data for all subgroups and comparison groups 



Numbers of test-takers by subgroup 



1999 through 2008: Grades 4, 8, and 10 math; grade 10 English 
language arts (ELA) 

2001 through 2008: Grade 6 math; grades 3, 4, and 7 ELA 

2006 through 2008: Grades 3, 5, and 7 math; grades 5, 6, and 8 ELA 

Cannot compute effect sizes; no mean scale scores or standard 
deviations available 

Massachusetts revised its definitions of racial/ethnic subgroups and 
advised that data can be considered comparable for racial/ethnic 
subgroup comparisons from 2005 through 2008. 

Not available for low-income students until 2005, for students who 
are nof low-income until 2006, or for English language learners 
(ELLs) until 2007. 

Students with disabilities and ELLs are compared with all students in 
the state because data are not available until 2007 for the 
comparison group of students who are nof disabled or for any 
year for students who are not ELLs. 

Available for 2007 through 2008 



Test Characteristics 

The characteristics highlighted below are for the state reading and mathematics tests used for accountability under the No Child Left Behind Act 
(NCLB). 



Test(s) used for NCLB accountability 



Massachusetts Comprehensive Assessment System (MCAS) 
MCAS Alternate Assessment (MCAS-Alt) 
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Grades tested for NCLB accountability 3-8, 10 in ELA and math, as of 2007 

State labels for achievement levels MA uses four achievement levels: Warning/Failing, Needs 

Improvement, Proficient, and Advanced/Above Proficient. For our 
analyses we treated Needs Improvement as Basic, Proficient as 
Proficient, and Advanced/Above Proficient as Advanced. 

High school NCLB test also used as an exit exam? Yes 

First year test used 1 998: Grades 4,8,10 in math; grade 1 0 in ELA 

2001 : Grades 3, 4, 7 in reading/ELA; grade 6 in math 
2006: Grades 3, 5, 7 in math; grades 5, 7, 8 in ELA 

Time of test administration Spring (opportunities for retests in fall, spring, and summer for 

students who did not pass the grade 10 test) 

Major changes in testing system (2002-present) 2002: New scaling system adopted 

2005-06: Reading/ELA and math tested in all of the grades 3-8 and 
1 0. Prior to 2005-06, reading/ELA was tested in grades 3, 4, 7, and 
10, and math was tested in grades 4, 6, 8, and 10. 

2006: Absent students without documented medical reasons counted 
as non-participants in testing; prior to 2006, counted as 
failing/warning 

2006: Test results reported on state Web site for both current and 
former limited-English-proficient (LEP) students; previously, only 
results for current LEP students were reported 
2006: Reporting of the “regular education” subgroup discontinued 

Comments In CEP’s 2008 achievement report, trends in reading/ELA were shown 

for grade 7. This was because reading/ELA tests were not 
administered in grade 8, the default grade for CEP’s analyses, until 
2006, which meant that only two years of grade 8 data were 
available. In this year’s tables and figures, trends in reading/ELA 
are shown for grade 8, the default grade, because three years of 
data are available, a long enough period to constitute a trend. 
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Overall Achievement — Percentages Proficient 

Figure MA-1. Percentage of Students Scoring at the Proficient Levei and Above in Reading 
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Table MA-1. Percentage of Students Scoring at the Proficient Level and Above in Reading 



Reporting Year Pre-NCLB Post-NCLB 

Grade Average Yearly Average Yearly 



Level 


1999 


2000 


2001 


2002 


2003 


2004 


2005 


2006 


2007 


2008 


Percentage Point Gain 
1999-2002^ 


Percentage Point Gain 
2002-2008^ 


Grade 3 






62% 


67% 


63% 


63% 


62% 


58% 


59% 


56% 


NA 


-1.8 


Grade 4 






51% 


54% 


56% 


56% 


50% 


50% 


56% 


49% 


NA 


-0.8 


Grade 5 
















59% 


63% 


61% 


NA 


1.0 


Grade 6 
















64% 


67% 


67% 


NA 


1.5 


Grade 7 






55% 


64% 


65% 


68% 


66% 


65% 


69% 


69% 


NA 


0.9 


Grade 8 
















74% 


75% 


75% 


NA 


0.5 


Grade 10 


34% 


36% 


50% 


60% 


61% 


63% 


64% 


70% 


71% 


75% 


8.5 


2.6 



Table reads: The percentage of 3'^'^ graders who scored at the proficient level and above on the state reading test increased from 62% in 2001 , to 67% in 2002, 
then dropped to 56% in 2008. The average yearly loss in the percentage proficient in grade 3 reading was 1 .8 percentage points per year after NCLB was enacted. 

^Averages are subject to rounding error. 
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Figure MA-2. Percentage of Students Scoring at the Proficient Levei and Above in Mathematics 
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Table MA-2. Percentage of Students Scoring at the Proficient Level and Above in Mathematics 



Reporting Year Pre-NCLB Post-NCLB 

Grade Average Yearly Average Yearly 



Level 


1999 


2000 


2001 


2002 


2003 


2004 


2005 


2006 


2007 


2008 


Percentage Point Gain 
1999-2002^ 


Percentage Point Gain 
2002-2008^ 


Grade 3 
















52% 


60% 


61% 


NA 


4.5 


Grade 4 


36% 


40% 


34% 


39% 


40% 


42% 


40% 


40% 


48% 


49% 


1.1 


1.6 


Grade 5 
















43% 


51% 


52% 


NA 


4.5 


Grade 6 






36% 


42% 


42% 


43% 


46% 


46% 


52% 


56% 


NA 


2.4 


Grade 7 
















40% 


46% 


47% 


NA 


3.5 


Grade 8 


28% 


34% 


34% 


34% 


37% 


39% 


39% 


40% 


45% 


49% 


1.9 


2.5 


Grade 10 


24% 


33% 


45% 


44% 


51% 


57% 


61% 


67% 


68% 


72% 


6.6 


4.7 



Table reads: The percentage of 3'^'^ graders who scored at the proficient level and above on the state math test increased from 52% in 2006 to 61% in 2008. The 
average yearly gain in the percentage proficient in grade 3 math was 4.5 percentage points per year after NCLB was enacted. 

^Averages are subject to rounding error. 
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Overall Achievement — Percentages Advanced, Proficient, and Basic 

How to read figures 3 and 4 and tables 3 and 4 

The stacked bars in figures 3 and 4 show the percentages of students scoring at the basic, proficient, and advanced levels on the state tests used 
for NCLB accountability. The following information may be helpful in interpreting the figures: 

• The percentage proficient and above — the benchmark used to determine adequate yearly progress under NCLB — is the sum of the middle 
and top segments of the bars (percentage proficient plus percentage advanced). 

• The percentage basic and above is the sum of all three segments of the bars (percentage basic plus percentage proficient plus 
percentage advanced). 

• The sums that result from adding the segments of the bars in these ways correspond with the percentages proficient and above, and basic 
and above, shown in tables 3 and 4. In a few instances, however, the sums in the figures may differ from those in the tables by a 
percentage point due to rounding. 

• The bars do not total 1 00% because students who score below the basic level are not displayed. 

• By looking at the percentages in each segment of the bars, one can see how achievement trends at the three levels interact. Ideally, one 
would want to see increases at all three levels, as more students move from below basic to basic achievement, from basic to proficient, 
and from proficient to advanced. But other scenarios may also be illuminating. For example, gains may occur in the percentage basic even 
if the percentage proficient and above has stayed the same, suggesting that progress has been made in moving students from the below 
basic to the basic level. Or, if the percentage proficient has grown while the percentages basic and advanced have shrunk, this suggests 
that educators may have focused a great deal of attention on moving students from the basic to proficient levels. 

• Some states use different labels for their achievement levels instead of basic, proficient, and advanced. The specific state labels are listed 
in the Test Characteristics section at the beginning of this profile. 
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Figure MA-3. Percentages of Students Scoring at the Advanced, Proficient, and Basic Leveis in Reading 
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Table MA-3. Percentages of Students Scoring at the Advanced, Proficient and Above, and Basic and Above Levels in Reading 



Achievement Level 






Reporting Year 








Average Yearly 
Percentage Point Galn^ 


2002 


2003 


2004 


2005 


2006 


2007 


2008 


Grade 4 


Advanced 


8% 


10% 


11% 


10% 


8% 


10% 


8% 


0.1 


Proficient and Above 


54% 


56% 


56% 


50% 


50% 


56% 


49% 


-0.8 


Basic and Above 


90% 


90% 


91% 


90% 


89% 


90% 


88% 


-0.4 


Grade 8 


Advanced 










12% 


12% 


12% 


0.0 


Proficient and Above 










74% 


75% 


75% 


0.5 


Basic and Above 










93% 


93% 


93% 


0.0 


Grade 10 


Advanced 


19% 


20% 


19% 


22% 


16% 


22% 


24% 


0.8 


Proficient and Above 


60% 


61% 


63% 


64% 


70% 


71% 


75% 


2.6 


Basic and Above 


86% 


89% 


90% 


89% 


93% 


95% 


96% 


1.6 



Table reads: The percentage of 4*^ graders who scored at the advanced level on their state reading test was 8% In 2002 and in 2008. Due to a rise and fall over 
these years, however, the average yearly gain in the percentage advanced was 0.1 percentage points per year in grade 4 reading. 

^Averages are subject to rounding error. 
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Figure MA-4. Percentages of Students Scoring at the Advanced, Proficient, and Basic Leveis in Mathematics 
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Table MA-4. Percentages of Students Scoring at the Advanced, Proficient and Above, and Basic and Above Levels in Mathematics 









Reporting Year 








Average Yearly 
Percentage Point Galn^ 


Achievement Level 


2002 


2003 


2004 


2005 


2006 


2007 


2008 


Grade 4 


Advanced 


12% 


12% 


14% 


14% 


15% 


19% 


20% 


1.3 


Proficient and Above 


39% 


40% 


42% 


40% 


40% 


48% 


49% 


1.6 


Basic and Above 


81% 


84% 


86% 


84% 


85% 


87% 


87% 


1.0 


Grade 8 


Advanced 


11% 


12% 


13% 


13% 


12% 


17% 


19% 


1.4 


Proficient and Above 


34% 


37% 


39% 


39% 


40% 


45% 


49% 


2.5 


Basic and Above 


67% 


67% 


71% 


69% 


71% 


75% 


76% 


1.5 


Grade 10 


Advanced 


20% 


24% 


29% 


34% 


40% 


41% 


42% 


3.7 


Proficient and Above 


44% 


51% 


57% 


61% 


67% 


68% 


72% 


4.7 


Basic and Above 


75% 


79% 


85% 


85% 


88% 


90% 


90% 


2.6 



Table reads: The percentage of 4*^ graders who scored at the advanced level on their state math test Increased from 12% in 2002 to 20% In 2008. During this 
period, the average yearly gain In the percentage advanced was 1.3 percentage points per year in grade 4 math. 

^Averages are subject to rounding error. 
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Key Terms 

Percentage proficient (and above) — The percentage of students in a group who score at and above the cut score for “proficient” performance on 
the state test used to determine progress under NCLB. The Act requires states to report student test performance in terms of at ieast three 
achievement levels: basic, proficient, and advanced. Adequate yearly progress determinations are based on the percentage of students scoring at 
the proficient level and above. 

Percentage basic (and above) — The percentage of students in a group who score at and above the cut score for “basic” performance on the 
state test used to determine progress under NCLB. 

Percentage advanced — The percentage of students in a group who reach or exceed the cut score for “advanced” performance on the state test 
used to determine progress under NCLB. 

Moderate-to-iarge gain — For the percentage basic, proficient, or advanced, an average gain of 1 or more percentage points per year. For effect 
size, an average gain of 0.02 or greater per year. 

Siightgain — For the percentage basic, proficient, or advanced, an average gain of less than 1 percentage point per year. For effect size, an 
average gain of less than 0.02 per year. 

Moderate-to-iarge deciine — For the percentage basic, proficient, or advanced, an average decline of 1 or more percentage points per year. For 
effect size, an average decline of 0.02 or greater per year. 

Siight deciine — For the percentage basic, proficient, or advanced, an average decline of less than 1 percentage points per year. For effect size, 
an average decline of less than 0.02 per year. 

Effect size — A statistical tool that conveys the amount of difference between test results using a common unit of measurement which does not 
depend on the scoring scale for a particular test. 

Accumuiated annuai effect size — The cumulative gain in effect size over a range of years. 

Mean scaie score — The arithmetical average of a group of test scores, expressed on a common scale for a particular state’s test. The mean is 
calculated by adding the scores and dividing the sum by the number of scores. 

Standard deviation — A measure of how much test scores tend to deviate from the mean — in other words, how spread out or bunched together 
test scores are. If students’ scores are bunched together, with many scores close to the mean, then the standard deviation will be small. If scores 
are spread out, with many students scoring at the high or low ends of the scale, then the standard deviation will be large. 
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Cautions and Explanations 

Different labels for achievement levels — For consistency, all of the state profiles developed for this report use a common set of labels (basic, 
proficient, and advanced) for the main achievement levels required by NCLB. In practice, however, some states may use different labels, such as 
“meets standard” instead of proficient, and some states have established additional achievement levels beyond those required by NCLB. 

Different names for subgroups — For the sake of consistency and ease of data tabulation, all of the state profiles developed for this report use a 
common set of names for the major student subgroups. In practice, however, states use various names for subgroups that may differ from those 
used here (such as using “Hispanic” instead of “Latino,” or “special education students” instead of “students with disabilities”). Moreover, a few 
states separately track the performance of subgroups not included in the analyses for this report. 

Special caution for students with disabilities and English language learners — T rends for students with disabilities and English language learners 
should be interpreted with caution because changes in federal guidance and state accountability plans may have altered which students in these 
subgroups are tested for accountability purposes, how they are tested, and when their test scores are counted as proficient under NCLB. These 
factors could affect the year-to-year comparability of test results. 

Inclusion of former English language learners — In many states, the subgroup of English language learners (also known as limited English 
proficient students) includes students who were formerly English language learners but who have achieved English language proficiency or 
fluency in the last two years. Federal NCLB regulations permit states to include these formerly ELL students (sometimes referred to as 
“redesignated fluent English proficient” students) in the ELL subgroup for up to two years for purposes of NCLB accountability. 

Limitations of percentage proficient measure — The percentage proficient, the main gauge of student performance under NCLB, can be easily 
understood and gives a snapshot of how many students have met their state’s performance expectations. But it also has several limitations as a 
measure of student achievement. Users of percentage proficient data should keep in mind these limitations, particularly the following: 

* “Proficient” means different things across different states. States vary widely in curriculum, learning expectations, and tests, and state tests differ 
considerably in their difficulty and cut scores for proficient performance. 

* Although this study has taken steps to avoid comparing test data where there have been “breaks” in comparability resulting from new tests, 
changes in content standards, revised cut scores, or other major changes in testing programs, the year-to-year comparability of test results in 
the same state may still be affected by less obvious policy and demographic changes. 

* Changes in student performance may occur that are not reflected in percentage proficient data, such as an increase in the number of students 
reaching performance levels below and above proficient (such as the basic or advanced levels). 

* The size of the achievement gaps between various subgroups depends in part on where a state sets its cut score for proficiency. For example, if 
a proficiency cut score is set so high that almost nobody reaches it or so low that almost everyone reaches it, there will be little apparent 
achievement gap. By contrast, if the cut score is closer to the mean test score, the gaps between subgroups will be more apparent. 

Difficulty of attributing causes — Although the tables above show trends in test scores since the enactment of NCLB, one cannot assume that 
these trends have occurred because of NCLB. It is always difficult to determine a cause-and-effect relationship between test score trends and any 
specific education policy or program due to the many federal, state, and local reforms undertaken in recent years and due to the lack of an 
appropriate “control” group of students not affected by NCLB. 





